Spatial-Intensity Transforms for Medical Image-to-Image Translation.

Image-to-image translation has seen major advances in computer vision but can be difficult to apply to medical images, where imaging artifacts and data scarcity degrade the performance of conditional generative adversarial networks. We develop the spatial-intensity transform (SIT) to improve output image quality while closely matching the target domain. SIT constrains the generator to a smooth spatial transform (diffeomorphism) composed with sparse intensity changes. SIT is a lightweight, modular network component that is effective on various architectures and training schemes. Relative to unconstrained baselines, this technique significantly improves image fidelity, and our models generalize robustly to different scanners. Additionally, SIT provides a disentangled view of anatomical and textural changes for each translation, making it easier to interpret the model's predictions in terms of physiological phenomena. We demonstrate SIT on two tasks: predicting longitudinal brain MRIs in patients with various stages of neurodegeneration, and visualizing changes with age and stroke severity in clinical brain scans of stroke patients. On the first task, our model accurately forecasts brain aging trajectories without supervised training on paired scans. On the second task, it captures associations between ventricle expansion and aging, as well as between white matter hyperintensities and stroke severity. As conditional generative models become increasingly versatile tools for visualization and forecasting, our approach demonstrates a simple and powerful technique for improving robustness, which is critical for translation to clinical settings. Source code is available at github.com/clintonjwang/spatial-intensity-transforms.

Radiomics-Derived Brain Age Predicts Functional Outcome After Acute Ischemic Stroke.

BACKGROUND AND OBJECTIVES: While chronological age is one of the most influential determinants of poststroke outcomes, little is known of the impact of neuroimaging-derived biological "brain age." We hypothesized that radiomics analyses of T2-FLAIR images texture would provide brain age estimates and that advanced brain age of patients with stroke will be associated with cardiovascular risk factors and worse functional outcomes. METHODS: We extracted radiomics from T2-FLAIR images acquired during acute stroke clinical evaluation. Brain age was determined from brain parenchyma radiomics using an ElasticNet linear regression model. Subsequently, relative brain age (RBA), which expresses brain age in comparison with chronological age-matched peers, was estimated. Finally, we built a linear regression model of RBA using clinical cardiovascular characteristics as inputs and a logistic regression model of favorable functional outcomes taking RBA as input. RESULTS: We reviewed 4,163 patients from a large multisite ischemic stroke cohort (mean age = 62.8 years, 42.0% female patients). T2-FLAIR radiomics predicted chronological ages (mean absolute error = 6.9 years, r = 0.81). After adjustment for covariates, RBA was higher and therefore described older-appearing brains in patients with hypertension, diabetes mellitus, a history of smoking, and a history of a prior stroke. In multivariate analyses, age, RBA, NIHSS, and a history of prior stroke were all significantly associated with functional outcome (respective adjusted odds ratios: 0.58, 0.76, 0.48, 0.55; all p-values < 0.001). Moreover, the negative effect of RBA on outcome was especially pronounced in minor strokes. DISCUSSION: T2-FLAIR radiomics can be used to predict brain age and derive RBA. Older-appearing brains, characterized by a higher RBA, reflect cardiovascular risk factor accumulation and are linked to worse outcomes after stroke.

Conventional vs. Drug-Eluting Beads Transarterial Chemoembolization for Unresectable Hepatocellular Carcinoma-A Propensity Score Weighted Comparison of Efficacy and Safety.

This study compared the efficacy and safety of conventional transarterial chemoembolization (cTACE) with drug-eluting beads (DEB)-TACE in patients with unresectable hepatocellular carcinoma (HCC). This retrospective analysis included 370 patients with HCC treated with cTACE (n = 248) or DEB-TACE (n = 122) (January 2000-July 2014). Overall survival (OS) was assessed using uni- and multivariate Cox proportional hazards models and Kaplan-Meier analysis. Additionally, baseline imaging was assessed, and clinical and laboratory toxicities were recorded. Propensity score weighting via a generalized boosted model was applied to account for group heterogeneity. There was no significant difference in OS between cTACE (20 months) and DEB-TACE patients (24.3 months, ratio 1.271, 95% confidence interval 0.876-1.69; p = 0.392). However, in patients with infiltrative disease, cTACE achieved longer OS (25.1 months) compared to DEB-TACE (9.2 months, ratio 0.366, 0.191-0.702; p = 0.003), whereas DEB-TACE proved more effective in nodular disease (39.4 months) than cTACE (18 months, ratio 0.458, 0.308-0681; p = 0.007). Adverse events occurred with similar frequency, except for abdominal pain, which was observed more frequently after DEB-TACE (101/116; 87.1%) than cTACE (119/157; 75.8%; p = 0.02). In conclusion, these findings suggest that tumor morphology and distribution should be used as parameters to inform decisions on the selection of embolic materials for TACE for a more personalized treatment planning in patients with unresectable HCC.

MRI Radiomic Signature of White Matter Hyperintensities Is Associated With Clinical Phenotypes.

OBJECTIVE: Neuroimaging measurements of brain structural integrity are thought to be surrogates for brain health, but precise assessments require dedicated advanced image acquisitions. By means of quantitatively describing conventional images, radiomic analyses hold potential for evaluating brain health. We sought to: (1) evaluate radiomics to assess brain structural integrity by predicting white matter hyperintensities burdens (WMH) and (2) uncover associations between predictive radiomic features and clinical phenotypes. METHODS: We analyzed a multi-site cohort of 4,163 acute ischemic strokes (AIS) patients with T2-FLAIR MR images with total brain and WMH segmentations. Radiomic features were extracted from normal-appearing brain tissue (brain mask-WMH mask). Radiomics-based prediction of personalized WMH burden was done using ElasticNet linear regression. We built a radiomic signature of WMH with stable selected features predictive of WMH burden and then related this signature to clinical variables using canonical correlation analysis (CCA). RESULTS: Radiomic features were predictive of WMH burden (R 2 = 0.855 ± 0.011). Seven pairs of canonical variates (CV) significantly correlated the radiomics signature of WMH and clinical traits with respective canonical correlations of 0.81, 0.65, 0.42, 0.24, 0.20, 0.15, and 0.15 (FDR-corrected p-values CV 1 - 6 < 0.001, p-value CV 7 = 0.012). The clinical CV1 was mainly influenced by age, CV2 by sex, CV3 by history of smoking and diabetes, CV4 by hypertension, CV5 by atrial fibrillation (AF) and diabetes, CV6 by coronary artery disease (CAD), and CV7 by CAD and diabetes. CONCLUSION: Radiomics extracted from T2-FLAIR images of AIS patients capture microstructural damage of the cerebral parenchyma and correlate with clinical phenotypes, suggesting different radiographical textural abnormalities per cardiovascular risk profile. Further research could evaluate radiomics to predict the progression of WMH and for the follow-up of stroke patients' brain health.

Deep learning-assisted differentiation of pathologically proven atypical and typical hepatocellular carcinoma (HCC) versus non-HCC on contrast-enhanced MRI of the liver.

OBJECTIVES: To train a deep learning model to differentiate between pathologically proven hepatocellular carcinoma (HCC) and non-HCC lesions including lesions with atypical imaging features on MRI. METHODS: This IRB-approved retrospective study included 118 patients with 150 lesions (93 (62%) HCC and 57 (38%) non-HCC) pathologically confirmed through biopsies (n = 72), resections (n = 29), liver transplants (n = 46), and autopsies (n = 3). Forty-seven percent of HCC lesions showed atypical imaging features (not meeting Liver Imaging Reporting and Data System [LI-RADS] criteria for definitive HCC/LR5). A 3D convolutional neural network (CNN) was trained on 140 lesions and tested for its ability to classify the 10 remaining lesions (5 HCC/5 non-HCC). Performance of the model was averaged over 150 runs with random sub-sampling to provide class-balanced test sets. A lesion grading system was developed to demonstrate the similarity between atypical HCC and non-HCC lesions prone to misclassification by the CNN. RESULTS: The CNN demonstrated an overall accuracy of 87.3%. Sensitivities/specificities for HCC and non-HCC lesions were 92.7%/82.0% and 82.0%/92.7%, respectively. The area under the receiver operating curve was 0.912. CNN's performance was correlated with the lesion grading system, becoming less accurate the more atypical imaging features the lesions showed. CONCLUSION: This study provides proof-of-concept for CNN-based classification of both typical- and atypical-appearing HCC lesions on multi-phasic MRI, utilizing pathologically confirmed lesions as "ground truth." KEY POINTS: • A CNN trained on atypical appearing pathologically proven HCC lesions not meeting LI-RADS criteria for definitive HCC (LR5) can correctly differentiate HCC lesions from other liver malignancies, potentially expanding the role of image-based diagnosis in primary liver cancer with atypical features. • The trained CNN demonstrated an overall accuracy of 87.3% and a computational time of < 3 ms which paves the way for clinical application as a decision support instrument.

Automated feature quantification of Lipiodol as imaging biomarker to predict therapeutic efficacy of conventional transarterial chemoembolization of liver cancer.

Conventional transarterial chemoembolization (cTACE) is a guideline-approved image-guided therapy option for liver cancer using the radiopaque drug-carrier and micro-embolic agent Lipiodol, which has been previously established as an imaging biomarker for tumor response. To establish automated quantitative and pattern-based image analysis techniques of Lipiodol deposition on 24 h post-cTACE CT as biomarker for treatment response. The density of Lipiodol deposits in 65 liver lesions was automatically quantified using Hounsfield Unit thresholds. Lipiodol deposition within the tumor was automatically assessed for patterns including homogeneity, sparsity, rim, and peripheral deposition. Lipiodol deposition was correlated with enhancing tumor volume (ETV) on baseline and follow-up MRI. ETV on baseline MRI strongly correlated with Lipiodol deposition on 24 h CT (p < 0.0001), with 8.22% ± 14.59 more Lipiodol in viable than necrotic tumor areas. On follow-up, tumor regions with Lipiodol showed higher rates of ETV reduction than areas without Lipiodol (p = 0.0475) and increasing densities of Lipiodol enhanced this effect. Also, homogeneous (p = 0.0006), non-sparse (p < 0.0001), rim deposition within sparse tumors (p = 0.045), and peripheral deposition (p < 0.0001) of Lipiodol showed improved response. This technical innovation study showed that an automated threshold-based volumetric feature characterization of Lipiodol deposits is feasible and enables practical use of Lipiodol as imaging biomarker for therapeutic efficacy after cTACE.

Spatial-Intensity Transform GANs for High Fidelity Medical Image-to-Image Translation.

Despite recent progress in image-to-image translation, it remains challenging to apply such techniques to clinical quality medical images. We develop a novel parameterization of conditional generative adversarial networks that achieves high image fidelity when trained to transform MRIs conditioned on a patient's age and disease severity. The spatial-intensity transform generative adversarial network (SIT-GAN) constrains the generator to a smooth spatial transform composed with sparse intensity changes. This technique improves image quality and robustness to artifacts, and generalizes to different scanners. We demonstrate SIT-GAN on a large clinical image dataset of stroke patients, where it captures associations between ventricle expansion and aging, as well as between white matter hyperintensities and stroke severity. Additionally, SIT-GAN provides a disentangled view of the variation in shape and appearance across subjects.

Deep learning for liver tumor diagnosis part II: convolutional neural network interpretation using radiologic imaging features.

OBJECTIVES: To develop a proof-of-concept "interpretable" deep learning prototype that justifies aspects of its predictions from a pre-trained hepatic lesion classifier. METHODS: A convolutional neural network (CNN) was engineered and trained to classify six hepatic tumor entities using 494 lesions on multi-phasic MRI, described in Part 1. A subset of each lesion class was labeled with up to four key imaging features per lesion. A post hoc algorithm inferred the presence of these features in a test set of 60 lesions by analyzing activation patterns of the pre-trained CNN model. Feature maps were generated that highlight regions in the original image that correspond to particular features. Additionally, relevance scores were assigned to each identified feature, denoting the relative contribution of a feature to the predicted lesion classification. RESULTS: The interpretable deep learning system achieved 76.5% positive predictive value and 82.9% sensitivity in identifying the correct radiological features present in each test lesion. The model misclassified 12% of lesions. Incorrect features were found more often in misclassified lesions than correctly identified lesions (60.4% vs. 85.6%). Feature maps were consistent with original image voxels contributing to each imaging feature. Feature relevance scores tended to reflect the most prominent imaging criteria for each class. CONCLUSIONS: This interpretable deep learning system demonstrates proof of principle for illuminating portions of a pre-trained deep neural network's decision-making, by analyzing inner layers and automatically describing features contributing to predictions. KEY POINTS: • An interpretable deep learning system prototype can explain aspects of its decision-making by identifying relevant imaging features and showing where these features are found on an image, facilitating clinical translation. • By providing feedback on the importance of various radiological features in performing differential diagnosis, interpretable deep learning systems have the potential to interface with standardized reporting systems such as LI-RADS, validating ancillary features and improving clinical practicality. • An interpretable deep learning system could potentially add quantitative data to radiologic reports and serve radiologists with evidence-based decision support.

Deep learning for liver tumor diagnosis part I: development of a convolutional neural network classifier for multi-phasic MRI.

OBJECTIVES: To develop and validate a proof-of-concept convolutional neural network (CNN)-based deep learning system (DLS) that classifies common hepatic lesions on multi-phasic MRI. METHODS: A custom CNN was engineered by iteratively optimizing the network architecture and training cases, finally consisting of three convolutional layers with associated rectified linear units, two maximum pooling layers, and two fully connected layers. Four hundred ninety-four hepatic lesions with typical imaging features from six categories were utilized, divided into training (n = 434) and test (n = 60) sets. Established augmentation techniques were used to generate 43,400 training samples. An Adam optimizer was used for training. Monte Carlo cross-validation was performed. After model engineering was finalized, classification accuracy for the final CNN was compared with two board-certified radiologists on an identical unseen test set. RESULTS: The DLS demonstrated a 92% accuracy, a 92% sensitivity (Sn), and a 98% specificity (Sp). Test set performance in a single run of random unseen cases showed an average 90% Sn and 98% Sp. The average Sn/Sp on these same cases for radiologists was 82.5%/96.5%. Results showed a 90% Sn for classifying hepatocellular carcinoma (HCC) compared to 60%/70% for radiologists. For HCC classification, the true positive and false positive rates were 93.5% and 1.6%, respectively, with a receiver operating characteristic area under the curve of 0.992. Computation time per lesion was 5.6 ms. CONCLUSION: This preliminary deep learning study demonstrated feasibility for classifying lesions with typical imaging features from six common hepatic lesion types, motivating future studies with larger multi-institutional datasets and more complex imaging appearances. KEY POINTS: • Deep learning demonstrates high performance in the classification of liver lesions on volumetric multi-phasic MRI, showing potential as an eventual decision-support tool for radiologists. • Demonstrating a classification runtime of a few milliseconds per lesion, a deep learning system could be incorporated into the clinical workflow in a time-efficient manner.

Slowing of contractile kinetics by myosin-binding protein C can be explained by its cooperative binding to the thin filament.

Cardiac myosin binding protein C (cMyBP-C) is a thick filament-associated protein that participates in the regulation of muscle contraction. Simplified in vitro systems show that cMyBP-C binds not only to myosin, but also to the actin filament. The physiological significance of these separate binding interactions remains unclear, as does the question of whether either interaction is capable of explaining the behavior of intact muscle from which cMyBP-C has been removed. We have used a computational model to explore the characteristic effects of myosin-binding versus actin-binding by cMyBP-C. Simulations suggest that myosin-cMyBP-C interactions reduce peak force and Ca2 + sensitivity of the myofilaments, but have no appreciable effect on the rate of force redevelopment (ktr). In contrast, cMyBP-C binding to actin increases myofilament Ca2 + sensitivity and slows ktrat sub-maximal Ca2 + values. This slowing is due to cooperation between cMyBP-C 'crossbridges' and traditional myosin crossbridges as they bind to and activate the actin thin filament. We further observed that an overall recapitulation of skinned myocardial data from wild type and cMyBP-C knockout mice requires the interaction of cMyBP-C with of both of its binding targets in our model. The assumption of significant interactions with both partners was also sufficient to explain published effects of cMyBP-C ablation on twitch kinetics. These modeling results strongly support the view that both binding interactions play critical roles in the physiology of intact muscle. Furthermore, they suggest that the widely observed phenomenon of slowed force development in the presence of cMyBP-C may actually be a manifestation of cooperative binding of this protein to the thin filament.